05 Error Metrics¶
[Estimated execution time: 15 min]
The toolkit allows to evaluate standard error metrics automatically. This notebook will continue the air quality data set used in the previous tutorial 04 Model Training.
In [1]:
import mogptk
import numpy as np
import pandas as pd
Air Quality MOGP¶
For this tutorial we will use the air quality dataset. The data set contains hourly averaged responses from an array of five metal oxide chemical sensors embedded in an Air Quality Chemical Multisensor Device. The device was located in the field in a significantly polluted area in an Italian city. Data were recorded for one year from March 2004 representing the longest freely available recordings of a deployed air quality chemical sensor device.
We will only use five columns: CO(GT), NMHC(GT), C6H6(GT), NOx(GT), NO2(GT). For more information on data loading check out the tutorial 01 Data Loading. For more information on data handling check out the tutorial 02 Data Preparation.
For each sensor the minimum value is -200, which is also the default value when there is an error in the measurements. We will ignore them by converting them to NaN.
In [2]:
df = pd.read_csv('data/AirQualityUCI.csv', delimiter=';')
# Replace missing values with NaN
df.replace(-200.0, np.nan, inplace=True)
# First two columns are date and time
# We convert it to a single column with datetime format
df['Date'] = pd.to_datetime(df['Date'] + ' ' + df['Time'], format='%d/%m/%Y %H.%M.%S')
# Define an initial date to compare all other to it
ini_date = pd.Timestamp('2004-03-10 00:00:00.0')
# Get elapsed hours
df['Time'] = (df['Date'] - ini_date) / pd.Timedelta(hours=1)
# Use only the first eight days of data
df2 = df[df['Date'] < pd.Timestamp('2004-03-19 00:00:00.0')]
Remove aditional data to simulate sensor failure. In this case for each channel we will first remove 50% of the observations and then remove complete sectors in order to get reconstructions from the other channels through learned cross correlations.
We will also use data transformation as defined in the data preparation and transformation tutorial 02 Data Preparation where each channel is normalized so it has zero mean and unit variance.
In [3]:
cols = ['CO(GT)', 'NMHC(GT)', 'C6H6(GT)', 'NOx(GT)', 'NO2(GT)']
dataset = mogptk.LoadDataFrame(df2, x_col='Time', y_col=cols)
for channel in dataset:
channel.remove_randomly(pct=0.5)
# drop relative ranges to simulate sensor failure
dataset[0].remove_relative_range(0.2, 0.3)
dataset[1].remove_relative_range(0.8, 1.0)
dataset[2].remove_relative_range(0.9, 1.0)
dataset[3].remove_relative_range(0.8, 1.0)
dataset[4].remove_relative_range(0.0, 0.2)
for channel in dataset:
channel.transform(mogptk.TransformDetrend(degree=1))
channel.transform(mogptk.TransformStandard())
dataset.plot();
In [4]:
method = 'Adam'
iters = 500
lr = 0.05
Independent channels with spectral mixture kernels¶
For each channel we will use four mixtures.
In [5]:
sm = mogptk.SM(dataset, Q=4)
sm.init_parameters('BNSE')
sm.train(method=method, lr=lr, iters=iters,
verbose=True, error='MAE', plot=True);
Starting optimization using Adam
‣ Model: SM
‣ Channels: 5
‣ Parameters: 65
‣ Training points: 402
‣ Initial loss: 503.226
‣ Initial error: 33.2931
Start Adam:
0/500 0:00:00 loss= 503.226 error= 33.2931
5/500 0:00:00 loss= 497.896 error= 33.9372
10/500 0:00:00 loss= 489.217 error= 33.0414
15/500 0:00:00 loss= 482.743 error= 33.5371
20/500 0:00:00 loss= 476.72 error= 32.7491
25/500 0:00:01 loss= 471.127 error= 33.3041
30/500 0:00:01 loss= 465.836 error= 32.6956
35/500 0:00:01 loss= 460.844 error= 32.8632
40/500 0:00:01 loss= 456.154 error= 32.8857
45/500 0:00:01 loss= 451.728 error= 32.892
50/500 0:00:02 loss= 447.548 error= 32.6396
55/500 0:00:02 loss= 443.593 error= 32.7133
60/500 0:00:02 loss= 439.867 error= 32.6249
65/500 0:00:02 loss= 436.374 error= 32.5903
70/500 0:00:03 loss= 433.106 error= 32.6223
75/500 0:00:03 loss= 430.056 error= 32.5878
80/500 0:00:03 loss= 427.222 error= 32.5342
85/500 0:00:04 loss= 424.593 error= 32.5085
90/500 0:00:04 loss= 422.161 error= 32.5081
95/500 0:00:04 loss= 419.917 error= 32.4625
100/500 0:00:04 loss= 417.983 error= 32.5256
105/500 0:00:05 loss= 416.003 error= 32.4347
110/500 0:00:05 loss= 414.251 error= 32.3536
115/500 0:00:05 loss= 412.657 error= 32.3897
120/500 0:00:05 loss= 411.209 error= 32.3412
125/500 0:00:06 loss= 409.899 error= 32.3658
130/500 0:00:06 loss= 408.72 error= 32.2846
135/500 0:00:06 loss= 407.654 error= 32.3208
140/500 0:00:06 loss= 406.692 error= 32.2682
145/500 0:00:07 loss= 405.829 error= 32.2485
150/500 0:00:07 loss= 405.057 error= 32.247
155/500 0:00:07 loss= 404.368 error= 32.2387
160/500 0:00:07 loss= 403.75 error= 32.2123
165/500 0:00:08 loss= 403.2 error= 32.1803
170/500 0:00:08 loss= 402.901 error= 32.1957
175/500 0:00:08 loss= 402.42 error= 32.1256
180/500 0:00:08 loss= 401.9 error= 32.1056
185/500 0:00:09 loss= 401.544 error= 32.0876
190/500 0:00:09 loss= 401.234 error= 32.1236
195/500 0:00:09 loss= 400.936 error= 32.0909
200/500 0:00:09 loss= 400.68 error= 32.0324
205/500 0:00:10 loss= 400.459 error= 32.0969
210/500 0:00:10 loss= 400.277 error= 32.072
215/500 0:00:10 loss= 400.04 error= 31.9964
220/500 0:00:10 loss= 399.919 error= 32.0881
225/500 0:00:11 loss= 399.694 error= 32.0356
230/500 0:00:11 loss= 399.483 error= 31.975
235/500 0:00:11 loss= 399.334 error= 31.9825
240/500 0:00:11 loss= 399.606 error= 32.0317
245/500 0:00:11 loss= 399.143 error= 31.9178
250/500 0:00:12 loss= 398.898 error= 31.9293
255/500 0:00:12 loss= 398.805 error= 32.0061
260/500 0:00:12 loss= 398.661 error= 31.9302
265/500 0:00:12 loss= 398.513 error= 31.9132
270/500 0:00:13 loss= 398.57 error= 31.9563
275/500 0:00:13 loss= 398.345 error= 31.8706
280/500 0:00:13 loss= 398.178 error= 31.8736
285/500 0:00:13 loss= 398.057 error= 31.8946
290/500 0:00:14 loss= 397.935 error= 31.8739
295/500 0:00:14 loss= 397.814 error= 31.782
300/500 0:00:14 loss= 397.733 error= 31.8633
305/500 0:00:14 loss= 397.584 error= 31.8412
310/500 0:00:15 loss= 397.479 error= 31.8405
315/500 0:00:15 loss= 397.378 error= 31.84
320/500 0:00:15 loss= 397.337 error= 31.8265
325/500 0:00:15 loss= 397.207 error= 31.8603
330/500 0:00:16 loss= 397.316 error= 31.6685
335/500 0:00:16 loss= 397.002 error= 31.7665
340/500 0:00:16 loss= 396.865 error= 31.7645
345/500 0:00:16 loss= 396.783 error= 31.7222
350/500 0:00:17 loss= 396.692 error= 31.7742
355/500 0:00:17 loss= 396.671 error= 31.6457
360/500 0:00:17 loss= 396.547 error= 31.7301
365/500 0:00:17 loss= 396.403 error= 31.699
370/500 0:00:18 loss= 396.284 error= 31.7633
375/500 0:00:18 loss= 396.223 error= 31.778
380/500 0:00:18 loss= 396.144 error= 31.79
385/500 0:00:19 loss= 396.023 error= 31.6705
390/500 0:00:19 loss= 395.98 error= 31.7925
395/500 0:00:19 loss= 396.179 error= 32.0185
400/500 0:00:19 loss= 395.874 error= 31.9131
405/500 0:00:20 loss= 395.726 error= 31.8448
410/500 0:00:20 loss= 395.577 error= 31.8701
415/500 0:00:20 loss= 395.511 error= 31.8069
420/500 0:00:21 loss= 395.479 error= 31.8941
425/500 0:00:21 loss= 395.543 error= 31.5627
430/500 0:00:21 loss= 395.235 error= 31.7371
435/500 0:00:21 loss= 395.114 error= 31.8124
440/500 0:00:21 loss= 395.045 error= 31.8659
445/500 0:00:22 loss= 394.96 error= 31.8842
450/500 0:00:22 loss= 394.871 error= 31.8631
455/500 0:00:22 loss= 394.765 error= 31.7916
460/500 0:00:23 loss= 394.698 error= 31.7108
465/500 0:00:23 loss= 394.971 error= 31.7429
470/500 0:00:23 loss= 394.641 error= 31.795
475/500 0:00:23 loss= 394.584 error= 31.6968
480/500 0:00:24 loss= 394.473 error= 31.8388
485/500 0:00:24 loss= 394.457 error= 31.59
490/500 0:00:24 loss= 394.352 error= 31.561
495/500 0:00:24 loss= 394.21 error= 31.8005
500/500 0:00:25 loss= 394.102 error= 31.8406
Finished
Optimization finished in 25.234 seconds
‣ Iterations: 500
‣ Final loss: 394.102
‣ Final error: 31.8406
Evaluating errors¶
Using the mogptk.error function given a test input and output it calculates:
- Mean absolute error (MAE)
- Mean absolute percentage error (MAPE)
- Root mean squared error (RMSE)
If only the raw values $y_{true} - y_{pred}$ are desired, the flag simple must be passed.
Multiple models for the same test set (X, Y) can be passed at once, the result will be a list with one element for each model passed, where each element is another list of length equal to the number or channels where each element is the error for said model and channel. This enables to obtain errors for multiple models for the same test set where each channel can have different numbers of test points.
In [6]:
sm.plot_prediction(title='SM on Air Quality Data');
In [7]:
mogptk.error(sm, disp=True, per_channel=False);
| MAE | MAPE | RMSE | |
|---|---|---|---|
| Name | |||
| SM | 31.840617 | 47.75155 | 58.443636 |
Multi Output Spectral Mixture (MOSM)¶
Next we use the multi output spectral mixture kernel (Parra et al, 2016).
In [8]:
mosm = mogptk.MOSM(dataset, Q=4)
mosm.init_parameters('BNSE')
mosm.train(method=method, lr=lr, iters=iters,
verbose=True, error='MAE', plot=True)
mosm.plot_prediction(title='MOSM on Air Quality Data');
Starting optimization using Adam
‣ Model: MOSM
‣ Channels: 5
‣ Parameters: 105
‣ Training points: 402
‣ Initial loss: 538.303
‣ Initial error: 42.0905
Start Adam:
0/500 0:00:00 loss= 538.303 error= 42.0905
5/500 0:00:00 loss= 546.787 error= 44.5344
10/500 0:00:00 loss= 540.718 error= 43.6076
15/500 0:00:01 loss= 534.985 error= 43.1139
20/500 0:00:01 loss= 531.09 error= 42.8474
25/500 0:00:02 loss= 526.55 error= 42.187
30/500 0:00:02 loss= 523.114 error= 41.9008
35/500 0:00:03 loss= 520.795 error= 41.7139
40/500 0:00:03 loss= 517.502 error= 41.3093
45/500 0:00:04 loss= 514.388 error= 41.0597
50/500 0:00:05 loss= 511.372 error= 40.7386
55/500 0:00:05 loss= 508.364 error= 40.4829
60/500 0:00:06 loss= 504.94 error= 40.2014
65/500 0:00:06 loss= 501.645 error= 39.9121
70/500 0:00:07 loss= 498.452 error= 39.6749
75/500 0:00:08 loss= 494.789 error= 39.4123
80/500 0:00:08 loss= 491.358 error= 39.1528
85/500 0:00:09 loss= 487.642 error= 38.9089
90/500 0:00:10 loss= 484.072 error= 38.7074
95/500 0:00:11 loss= 480.332 error= 38.4734
100/500 0:00:12 loss= 476.73 error= 38.2615
105/500 0:00:13 loss= 473.164 error= 38.1123
110/500 0:00:13 loss= 469.698 error= 37.9166
115/500 0:00:14 loss= 466.379 error= 37.7805
120/500 0:00:15 loss= 463.516 error= 37.7519
125/500 0:00:16 loss= 460.28 error= 37.5855
130/500 0:00:17 loss= 457.34 error= 37.4908
135/500 0:00:17 loss= 454.585 error= 37.4072
140/500 0:00:18 loss= 452.003 error= 37.32
145/500 0:00:19 loss= 449.619 error= 37.2138
150/500 0:00:20 loss= 447.644 error= 37.2355
155/500 0:00:21 loss= 445.339 error= 37.1561
160/500 0:00:21 loss= 443.414 error= 37.1142
165/500 0:00:22 loss= 441.683 error= 37.0765
170/500 0:00:23 loss= 439.972 error= 37.066
175/500 0:00:24 loss= 438.45 error= 37.0508
180/500 0:00:24 loss= 437.189 error= 37.0065
185/500 0:00:25 loss= 435.684 error= 37.0052
190/500 0:00:26 loss= 434.47 error= 37.0528
195/500 0:00:27 loss= 433.592 error= 37.0057
200/500 0:00:28 loss= 432.205 error= 36.9967
205/500 0:00:29 loss= 431.362 error= 36.9996
210/500 0:00:30 loss= 430.342 error= 37.0004
215/500 0:00:31 loss= 429.532 error= 36.9872
220/500 0:00:31 loss= 428.85 error= 36.985
225/500 0:00:32 loss= 428.005 error= 36.9951
230/500 0:00:33 loss= 427.383 error= 36.9939
235/500 0:00:34 loss= 426.61 error= 37.0344
240/500 0:00:35 loss= 426.427 error= 37.0018
245/500 0:00:36 loss= 425.396 error= 36.9854
250/500 0:00:37 loss= 424.878 error= 36.9649
255/500 0:00:37 loss= 424.588 error= 37.0682
260/500 0:00:38 loss= 424.192 error= 36.9702
265/500 0:00:38 loss= 423.294 error= 36.9399
270/500 0:00:39 loss= 423.203 error= 37.046
275/500 0:00:40 loss= 423.07 error= 37.0962
280/500 0:00:40 loss= 422.045 error= 36.9919
285/500 0:00:41 loss= 421.647 error= 36.983
290/500 0:00:41 loss= 421.352 error= 37.006
295/500 0:00:42 loss= 421.05 error= 36.9945
300/500 0:00:43 loss= 420.935 error= 36.9453
305/500 0:00:44 loss= 420.095 error= 36.908
310/500 0:00:45 loss= 420.057 error= 37.0039
315/500 0:00:45 loss= 420.381 error= 37.172
320/500 0:00:46 loss= 419.304 error= 37.0697
325/500 0:00:47 loss= 419.062 error= 37.0237
330/500 0:00:48 loss= 418.795 error= 37.0373
335/500 0:00:48 loss= 418.543 error= 37.0972
340/500 0:00:49 loss= 418.262 error= 37.0616
345/500 0:00:50 loss= 418.088 error= 37.0067
350/500 0:00:51 loss= 417.784 error= 37.052
355/500 0:00:52 loss= 417.329 error= 37.0571
360/500 0:00:53 loss= 417.962 error= 37.248
365/500 0:00:55 loss= 417.751 error= 37.148
370/500 0:00:56 loss= 416.983 error= 37.0826
375/500 0:00:57 loss= 416.665 error= 37.0765
380/500 0:00:57 loss= 416.702 error= 37.0803
385/500 0:00:58 loss= 416.452 error= 37.0439
390/500 0:01:00 loss= 416.059 error= 37.0657
395/500 0:01:01 loss= 416.317 error= 37.3335
400/500 0:01:02 loss= 416.149 error= 37.2948
405/500 0:01:03 loss= 416.153 error= 37.2517
410/500 0:01:04 loss= 415.959 error= 37.2569
415/500 0:01:05 loss= 416.404 error= 37.2878
420/500 0:01:06 loss= 416.618 error= 37.3746
425/500 0:01:07 loss= 418.033 error= 37.078
430/500 0:01:07 loss= 416.969 error= 37.4217
435/500 0:01:08 loss= 417.138 error= 37.2567
440/500 0:01:09 loss= 418.292 error= 37.559
445/500 0:01:10 loss= 416.931 error= 37.1602
450/500 0:01:11 loss= 415.315 error= 37.4222
455/500 0:01:12 loss= 415.624 error= 37.5189
460/500 0:01:13 loss= 415.115 error= 37.2232
465/500 0:01:13 loss= 415.208 error= 37.1509
470/500 0:01:14 loss= 415.071 error= 37.3978
475/500 0:01:15 loss= 414.42 error= 37.149
480/500 0:01:16 loss= 414.266 error= 37.3492
485/500 0:01:17 loss= 414.003 error= 37.1281
490/500 0:01:18 loss= 413.718 error= 37.2468
495/500 0:01:18 loss= 413.784 error= 37.3052
500/500 0:01:19 loss= 413.58 error= 37.1608
Finished
Optimization finished in 1 minute 19 seconds
‣ Iterations: 500
‣ Final loss: 413.58
‣ Final error: 37.1608
Cross Spectral Mixture (CSM)¶
Then we use the cross spectral mixture kernel (Ulrich et al, 2015).
In [9]:
csm = mogptk.CSM(dataset, Q=4)
csm.init_parameters()
csm.train(method=method, lr=lr, iters=iters,
verbose=True, error='MAE', plot=True)
csm.plot_prediction(title='CSM on Air Quality Data');
Starting optimization using Adam
‣ Model: CSM
‣ Channels: 5
‣ Parameters: 53
‣ Training points: 402
‣ Initial loss: 473.578
‣ Initial error: 28.4435
Start Adam:
0/500 0:00:00 loss= 473.578 error= 28.4435
5/500 0:00:02 loss= 462.267 error= 28.5251
10/500 0:00:04 loss= 452.686 error= 29.2235
15/500 0:00:06 loss= 447.663 error= 29.157
20/500 0:00:09 loss= 441.75 error= 28.4208
25/500 0:00:11 loss= 435.853 error= 28.9513
30/500 0:00:13 loss= 430.834 error= 28.8211
35/500 0:00:15 loss= 425.952 error= 28.7363
40/500 0:00:17 loss= 421.358 error= 28.6236
45/500 0:00:19 loss= 417.136 error= 28.743
50/500 0:00:21 loss= 413.228 error= 28.7319
55/500 0:00:23 loss= 409.583 error= 28.707
60/500 0:00:25 loss= 406.229 error= 28.588
65/500 0:00:27 loss= 403.133 error= 28.5921
70/500 0:00:29 loss= 400.274 error= 28.5655
75/500 0:00:31 loss= 397.653 error= 28.5313
80/500 0:00:33 loss= 395.247 error= 28.52
85/500 0:00:36 loss= 393.048 error= 28.4893
90/500 0:00:38 loss= 391.037 error= 28.4821
95/500 0:00:40 loss= 389.204 error= 28.4498
100/500 0:00:42 loss= 387.532 error= 28.4391
105/500 0:00:44 loss= 386.011 error= 28.4216
110/500 0:00:46 loss= 384.627 error= 28.4105
115/500 0:00:48 loss= 383.369 error= 28.3959
120/500 0:00:50 loss= 382.225 error= 28.3788
125/500 0:00:52 loss= 381.186 error= 28.3663
130/500 0:00:53 loss= 380.242 error= 28.3581
135/500 0:00:55 loss= 379.383 error= 28.3486
140/500 0:00:56 loss= 378.601 error= 28.3382
145/500 0:00:57 loss= 377.889 error= 28.3314
150/500 0:00:58 loss= 377.24 error= 28.3243
155/500 0:00:59 loss= 376.651 error= 28.3295
160/500 0:01:00 loss= 376.165 error= 28.267
165/500 0:01:01 loss= 375.681 error= 28.3391
170/500 0:01:02 loss= 375.153 error= 28.2708
175/500 0:01:03 loss= 374.73 error= 28.2717
180/500 0:01:04 loss= 374.344 error= 28.2795
185/500 0:01:05 loss= 373.967 error= 28.2587
190/500 0:01:06 loss= 373.68 error= 28.2476
195/500 0:01:07 loss= 373.423 error= 28.2499
200/500 0:01:08 loss= 373.104 error= 28.2445
205/500 0:01:09 loss= 372.791 error= 28.2271
210/500 0:01:10 loss= 372.504 error= 28.227
215/500 0:01:11 loss= 372.235 error= 28.2284
220/500 0:01:12 loss= 371.987 error= 28.2253
225/500 0:01:13 loss= 371.772 error= 28.2254
230/500 0:01:15 loss= 371.564 error= 28.2211
235/500 0:01:16 loss= 371.374 error= 28.2211
240/500 0:01:17 loss= 371.195 error= 28.2194
245/500 0:01:18 loss= 371.089 error= 28.2226
250/500 0:01:19 loss= 371.196 error= 28.1996
255/500 0:01:20 loss= 370.782 error= 28.2092
260/500 0:01:22 loss= 370.565 error= 28.2283
265/500 0:01:23 loss= 370.462 error= 28.2416
270/500 0:01:24 loss= 370.34 error= 28.2376
275/500 0:01:25 loss= 370.185 error= 28.2165
280/500 0:01:27 loss= 370.101 error= 28.2035
285/500 0:01:28 loss= 369.986 error= 28.2254
290/500 0:01:29 loss= 369.887 error= 28.2163
295/500 0:01:30 loss= 369.89 error= 28.2491
300/500 0:01:31 loss= 370.359 error= 28.1823
305/500 0:01:32 loss= 369.907 error= 28.1886
310/500 0:01:33 loss= 369.712 error= 28.2032
315/500 0:01:34 loss= 369.589 error= 28.2144
320/500 0:01:36 loss= 369.486 error= 28.2176
325/500 0:01:37 loss= 369.398 error= 28.2244
330/500 0:01:39 loss= 369.32 error= 28.2307
335/500 0:01:40 loss= 369.255 error= 28.2344
340/500 0:01:42 loss= 369.212 error= 28.2402
345/500 0:01:43 loss= 369.163 error= 28.2419
350/500 0:01:44 loss= 369.113 error= 28.2347
355/500 0:01:45 loss= 369.066 error= 28.2349
360/500 0:01:46 loss= 369.034 error= 28.2408
365/500 0:01:47 loss= 369.104 error= 28.2177
370/500 0:01:48 loss= 369.805 error= 28.304
375/500 0:01:49 loss= 369.291 error= 28.2803
380/500 0:01:51 loss= 369.098 error= 28.2817
385/500 0:01:53 loss= 368.984 error= 28.2697
390/500 0:01:54 loss= 368.898 error= 28.269
395/500 0:01:56 loss= 368.834 error= 28.2649
400/500 0:01:57 loss= 368.787 error= 28.2561
405/500 0:01:59 loss= 368.753 error= 28.2602
410/500 0:02:01 loss= 368.723 error= 28.2559
415/500 0:02:03 loss= 368.69 error= 28.2565
420/500 0:02:05 loss= 368.66 error= 28.2544
425/500 0:02:08 loss= 368.634 error= 28.2523
430/500 0:02:10 loss= 368.61 error= 28.254
435/500 0:02:12 loss= 368.589 error= 28.2507
440/500 0:02:14 loss= 368.569 error= 28.2518
445/500 0:02:16 loss= 368.568 error= 28.2608
450/500 0:02:18 loss= 368.83 error= 28.2305
455/500 0:02:20 loss= 368.925 error= 28.2834
460/500 0:02:23 loss= 368.81 error= 28.2869
465/500 0:02:25 loss= 368.657 error= 28.2848
470/500 0:02:27 loss= 368.557 error= 28.2857
475/500 0:02:29 loss= 368.494 error= 28.2827
480/500 0:02:32 loss= 368.452 error= 28.277
485/500 0:02:34 loss= 368.416 error= 28.2743
490/500 0:02:37 loss= 368.387 error= 28.2729
495/500 0:02:39 loss= 368.36 error= 28.2691
500/500 0:02:42 loss= 368.342 error= 28.2634
Finished
Optimization finished in 2 minutes 42 seconds
‣ Iterations: 500
‣ Final loss: 368.342
‣ Final error: 28.2634
Spectral Mixture - Linear Model of Corregionalization (SM-LMC)¶
Lastly we fit the spectral mixture linear model of corregionalization (Wilson, 2014).
In [10]:
smlmc = mogptk.SM_LMC(dataset, Q=4)
smlmc.init_parameters()
smlmc.train(method=method, lr=lr, iters=iters,
verbose=True, error='MAE', plot=True)
smlmc.plot_prediction(title='SM-LMC on Air Quality Data');
Starting optimization using Adam
‣ Model: SM-LMC
‣ Channels: 5
‣ Parameters: 33
‣ Training points: 402
‣ Initial loss: 471.894
‣ Initial error: 27.767
Start Adam:
0/500 0:00:00 loss= 471.894 error= 27.767
5/500 0:00:00 loss= 463.752 error= 27.9928
10/500 0:00:01 loss= 456.346 error= 28.0933
15/500 0:00:01 loss= 449.342 error= 28.1606
20/500 0:00:02 loss= 443.058 error= 28.5212
25/500 0:00:02 loss= 437.238 error= 28.4841
30/500 0:00:03 loss= 431.349 error= 28.2738
35/500 0:00:03 loss= 426.032 error= 28.3295
40/500 0:00:04 loss= 420.709 error= 28.3723
45/500 0:00:05 loss= 415.791 error= 28.3594
50/500 0:00:05 loss= 411.02 error= 28.368
55/500 0:00:06 loss= 406.518 error= 28.3358
60/500 0:00:06 loss= 402.237 error= 28.3715
65/500 0:00:07 loss= 398.174 error= 28.3898
70/500 0:00:08 loss= 394.216 error= 28.4864
75/500 0:00:08 loss= 390.261 error= 28.5489
80/500 0:00:09 loss= 386.495 error= 28.4288
85/500 0:00:09 loss= 383.012 error= 28.3982
90/500 0:00:10 loss= 379.82 error= 28.3663
95/500 0:00:10 loss= 376.864 error= 28.3116
100/500 0:00:11 loss= 374.101 error= 28.3491
105/500 0:00:11 loss= 371.581 error= 28.3249
110/500 0:00:12 loss= 369.296 error= 28.262
115/500 0:00:13 loss= 367.21 error= 28.245
120/500 0:00:13 loss= 365.33 error= 28.2108
125/500 0:00:14 loss= 363.633 error= 28.1621
130/500 0:00:15 loss= 362.109 error= 28.124
135/500 0:00:15 loss= 360.738 error= 28.0959
140/500 0:00:16 loss= 359.509 error= 28.0657
145/500 0:00:16 loss= 358.406 error= 28.0306
150/500 0:00:17 loss= 357.414 error= 27.9978
155/500 0:00:17 loss= 356.519 error= 27.9688
160/500 0:00:18 loss= 355.708 error= 27.9401
165/500 0:00:18 loss= 354.969 error= 27.9086
170/500 0:00:19 loss= 354.291 error= 27.8743
175/500 0:00:20 loss= 353.662 error= 27.8369
180/500 0:00:20 loss= 353.074 error= 27.7965
185/500 0:00:21 loss= 352.519 error= 27.7524
190/500 0:00:21 loss= 351.992 error= 27.7064
195/500 0:00:22 loss= 351.487 error= 27.66
200/500 0:00:23 loss= 351.005 error= 27.616
205/500 0:00:23 loss= 350.55 error= 27.5489
210/500 0:00:24 loss= 350.06 error= 27.5107
215/500 0:00:25 loss= 349.543 error= 27.4326
220/500 0:00:25 loss= 349.014 error= 27.3409
225/500 0:00:26 loss= 348.39 error= 27.2457
230/500 0:00:26 loss= 347.655 error= 27.0773
235/500 0:00:27 loss= 346.856 error= 26.923
240/500 0:00:28 loss= 346.05 error= 26.7454
245/500 0:00:29 loss= 345.28 error= 26.6056
250/500 0:00:29 loss= 344.571 error= 26.4746
255/500 0:00:30 loss= 343.935 error= 26.365
260/500 0:00:30 loss= 343.374 error= 26.2718
265/500 0:00:31 loss= 342.878 error= 26.1882
270/500 0:00:32 loss= 342.441 error= 26.1184
275/500 0:00:32 loss= 342.052 error= 26.0604
280/500 0:00:33 loss= 341.703 error= 26.0114
285/500 0:00:34 loss= 341.384 error= 25.9699
290/500 0:00:34 loss= 341.09 error= 25.9345
295/500 0:00:35 loss= 340.815 error= 25.9045
300/500 0:00:36 loss= 340.553 error= 25.8789
305/500 0:00:36 loss= 340.3 error= 25.8565
310/500 0:00:37 loss= 340.054 error= 25.8379
315/500 0:00:37 loss= 339.81 error= 25.8249
320/500 0:00:38 loss= 339.566 error= 25.81
325/500 0:00:38 loss= 339.396 error= 25.824
330/500 0:00:39 loss= 339.071 error= 25.813
335/500 0:00:40 loss= 338.873 error= 25.8221
340/500 0:00:40 loss= 338.502 error= 25.8026
345/500 0:00:41 loss= 338.235 error= 25.7953
350/500 0:00:41 loss= 337.914 error= 25.8093
355/500 0:00:42 loss= 337.616 error= 25.8053
360/500 0:00:42 loss= 337.327 error= 25.7907
365/500 0:00:43 loss= 337.051 error= 25.7849
370/500 0:00:43 loss= 336.794 error= 25.7821
375/500 0:00:44 loss= 336.558 error= 25.7796
380/500 0:00:45 loss= 336.337 error= 25.7734
385/500 0:00:45 loss= 336.135 error= 25.7671
390/500 0:00:46 loss= 335.947 error= 25.7605
395/500 0:00:46 loss= 335.769 error= 25.7536
400/500 0:00:47 loss= 335.601 error= 25.7457
405/500 0:00:48 loss= 335.439 error= 25.7358
410/500 0:00:48 loss= 335.281 error= 25.7232
415/500 0:00:49 loss= 335.124 error= 25.7077
420/500 0:00:49 loss= 334.965 error= 25.6877
425/500 0:00:50 loss= 334.8 error= 25.6606
430/500 0:00:51 loss= 334.618 error= 25.6211
435/500 0:00:51 loss= 334.403 error= 25.5548
440/500 0:00:52 loss= 334.058 error= 25.3722
445/500 0:00:53 loss= 331.974 error= 24.1401
450/500 0:00:54 loss= 323.693 error= 23.621
455/500 0:00:55 loss= 321.113 error= 23.2902
460/500 0:00:56 loss= 317.299 error= 23.0304
465/500 0:00:57 loss= 314.528 error= 22.718
470/500 0:00:58 loss= 312.673 error= 22.4864
475/500 0:00:59 loss= 311.368 error= 22.3612
480/500 0:01:00 loss= 310.393 error= 22.2992
485/500 0:01:01 loss= 309.647 error= 22.2548
490/500 0:01:01 loss= 309.041 error= 22.2114
495/500 0:01:02 loss= 308.531 error= 22.1914
500/500 0:01:02 loss= 308.087 error= 22.1675
Finished
Optimization finished in 1 minute 2 seconds
‣ Iterations: 500
‣ Final loss: 308.087
‣ Final error: 22.1675
Convolutional Gaussian (CONV)¶
In [11]:
conv = mogptk.CONV(dataset, Q=4)
conv.init_parameters()
conv.train(method=method, lr=lr, iters=iters,
verbose=True, error='MAE', plot=True)
conv.plot_prediction(title='CONV on Air Quality Data');
Starting optimization using Adam
‣ Model: CONV
‣ Channels: 5
‣ Parameters: 49
‣ Training points: 402
‣ Initial loss: 514.873
‣ Initial error: 31.721
Start Adam:
0/500 0:00:00 loss= 514.873 error= 31.721
5/500 0:00:01 loss= 508.99 error= 30.9761
10/500 0:00:03 loss= 503.065 error= 30.2364
15/500 0:00:05 loss= 497.097 error= 29.502
20/500 0:00:07 loss= 491.082 error= 28.7707
25/500 0:00:10 loss= 485.019 error= 28.04
30/500 0:00:12 loss= 478.906 error= 27.3331
35/500 0:00:14 loss= 472.74 error= 26.6875
40/500 0:00:16 loss= 466.519 error= 26.0872
45/500 0:00:17 loss= 460.243 error= 25.5193
50/500 0:00:19 loss= 453.912 error= 24.9642
55/500 0:00:20 loss= 447.532 error= 24.4283
60/500 0:00:23 loss= 441.108 error= 23.922
65/500 0:00:25 loss= 434.647 error= 23.4217
70/500 0:00:27 loss= 428.16 error= 22.9263
75/500 0:00:29 loss= 421.658 error= 22.4357
80/500 0:00:32 loss= 415.158 error= 21.9642
85/500 0:00:34 loss= 408.677 error= 21.5079
90/500 0:00:35 loss= 402.234 error= 21.0648
95/500 0:00:36 loss= 395.85 error= 20.6434
100/500 0:00:38 loss= 389.545 error= 20.2558
105/500 0:00:39 loss= 383.337 error= 19.8972
110/500 0:00:40 loss= 377.246 error= 19.5717
115/500 0:00:42 loss= 371.288 error= 19.2781
120/500 0:00:44 loss= 365.479 error= 19.0106
125/500 0:00:46 loss= 359.83 error= 18.7768
130/500 0:00:47 loss= 354.351 error= 18.571
135/500 0:00:49 loss= 349.047 error= 18.3804
140/500 0:00:50 loss= 343.923 error= 18.2142
145/500 0:00:52 loss= 338.978 error= 18.064
150/500 0:00:54 loss= 334.207 error= 17.9298
155/500 0:00:57 loss= 329.602 error= 17.804
160/500 0:00:59 loss= 325.143 error= 17.684
165/500 0:01:01 loss= 320.798 error= 17.5579
170/500 0:01:04 loss= 316.521 error= 17.4178
175/500 0:01:06 loss= 312.262 error= 17.25
180/500 0:01:07 loss= 307.993 error= 17.0495
185/500 0:01:09 loss= 303.725 error= 16.8221
190/500 0:01:11 loss= 299.501 error= 16.5971
195/500 0:01:12 loss= 295.383 error= 16.4077
200/500 0:01:14 loss= 291.42 error= 16.2511
205/500 0:01:19 loss= 287.642 error= 16.1157
210/500 0:01:24 loss= 284.05 error= 15.9991
215/500 0:01:30 loss= 280.626 error= 15.9052
220/500 0:01:34 loss= 277.347 error= 15.8252
225/500 0:01:37 loss= 274.205 error= 15.7565
230/500 0:01:39 loss= 271.203 error= 15.6895
235/500 0:01:42 loss= 268.346 error= 15.6238
240/500 0:01:44 loss= 265.635 error= 15.5609
245/500 0:01:46 loss= 263.066 error= 15.5009
250/500 0:01:47 loss= 260.631 error= 15.4437
255/500 0:01:48 loss= 258.325 error= 15.3906
260/500 0:01:50 loss= 256.142 error= 15.3388
265/500 0:01:51 loss= 254.075 error= 15.29
270/500 0:01:52 loss= 252.115 error= 15.2524
275/500 0:01:53 loss= 250.255 error= 15.2156
280/500 0:01:55 loss= 248.484 error= 15.1785
285/500 0:01:56 loss= 246.793 error= 15.1411
290/500 0:01:57 loss= 245.172 error= 15.1031
295/500 0:01:58 loss= 243.611 error= 15.0645
300/500 0:02:00 loss= 242.099 error= 15.0229
305/500 0:02:01 loss= 240.627 error= 14.9783
310/500 0:02:02 loss= 239.185 error= 14.9295
315/500 0:02:04 loss= 237.764 error= 14.8755
320/500 0:02:05 loss= 236.358 error= 14.8168
325/500 0:02:06 loss= 234.961 error= 14.7552
330/500 0:02:08 loss= 233.57 error= 14.6913
335/500 0:02:09 loss= 232.186 error= 14.6225
340/500 0:02:11 loss= 230.809 error= 14.5496
345/500 0:02:12 loss= 229.445 error= 14.4756
350/500 0:02:13 loss= 228.096 error= 14.3995
355/500 0:02:15 loss= 226.77 error= 14.3221
360/500 0:02:16 loss= 225.47 error= 14.2464
365/500 0:02:17 loss= 224.199 error= 14.1753
370/500 0:02:19 loss= 222.959 error= 14.1111
375/500 0:02:20 loss= 221.75 error= 14.052
380/500 0:02:21 loss= 220.57 error= 13.9998
385/500 0:02:23 loss= 219.421 error= 13.9595
390/500 0:02:24 loss= 218.3 error= 13.9282
395/500 0:02:25 loss= 217.208 error= 13.9
400/500 0:02:27 loss= 216.145 error= 13.8727
405/500 0:02:28 loss= 215.11 error= 13.8448
410/500 0:02:29 loss= 214.101 error= 13.8151
415/500 0:02:30 loss= 213.117 error= 13.7857
420/500 0:02:32 loss= 212.158 error= 13.7564
425/500 0:02:33 loss= 211.223 error= 13.7277
430/500 0:02:34 loss= 210.31 error= 13.701
435/500 0:02:35 loss= 209.419 error= 13.6745
440/500 0:02:36 loss= 208.549 error= 13.6479
445/500 0:02:38 loss= 207.699 error= 13.621
450/500 0:02:39 loss= 206.869 error= 13.5935
455/500 0:02:40 loss= 206.059 error= 13.5653
460/500 0:02:42 loss= 205.267 error= 13.5362
465/500 0:02:43 loss= 204.494 error= 13.5063
470/500 0:02:44 loss= 203.74 error= 13.4757
475/500 0:02:46 loss= 203.004 error= 13.4447
480/500 0:02:47 loss= 202.288 error= 13.4134
485/500 0:02:48 loss= 201.591 error= 13.3824
490/500 0:02:49 loss= 200.913 error= 13.3521
495/500 0:02:51 loss= 200.256 error= 13.3227
500/500 0:02:52 loss= 199.619 error= 13.2947
Finished
Optimization finished in 2 minutes 52 seconds
‣ Iterations: 500
‣ Final loss: 199.619
‣ Final error: 13.2947
Compare errors¶
We will take the mean MAE, MAPE, and RMSE for all the channels and compare all the models.
In [12]:
mogptk.error(sm, mosm, csm, smlmc, conv, disp=True)
| MAE | MAPE | RMSE | |
|---|---|---|---|
| Name | |||
| SM | 31.840617 | 47.751550 | 58.443636 |
| MOSM | 37.160805 | 65.353113 | 71.388803 |
| CSM | 28.263449 | 42.516146 | 53.322366 |
| SM-LMC | 22.167511 | 30.401371 | 43.084923 |
| CONV | 13.294705 | 18.211582 | 25.173724 |